Load distribution for integration scenarios

ABSTRACT

The disclosure generally describes methods, software, and systems for handling integration flows. A set of flows is initially deployed to a single worker set. A load balancing issue is identified that is associated with initial runtime interactions by workers with the single worker set. In response to identifying the load balancing issue, the load balancing issue is analyzed to determine whether to autoscale or generate a new worker set. Load balancing is performed to initiate at least one new worker set. At least one flow to be moved to the at least one new worker set is identified. Movement of the identified at least one flow from a current worker set to a new worker set is performed.

SUMMARY

This disclosure generally describes computer-implemented methods,software, and systems for assigning and managing workflows, includingmanaging load distribution for integration scenarios. Onecomputer-implemented method includes: deploying a set of flows initiallyto a single worker set; identifying a load balancing issue associatedwith initial runtime interactions by workers with the single worker set;in response to identifying the load balancing issue, analyzing the loadbalancing issue to determine whether to autoscale or generate a newworker set; performing load balancing to initiate at least one newworker set; identifying at least one flow to be moved to the at leastone new worker set; and performing movement of the identified at leastone flow from a current worker set to a new worker set.

The foregoing and other implementations can each optionally include oneor more of the following features, alone or in combination. Inparticular, one implementation can include all the following features:

In a first aspect, combinable with any of the previous aspects, whereinload balancing issues include out-of-memory exceptions.

In a second aspect, combinable with any of the previous aspects, whereinidentifying at least one flow to be moved to the at least one new workerset includes randomly moving the at least one flow.

In a third aspect, combinable with any of the previous aspects, whereinidentifying at least one flow to be moved to the at least one new workerset includes: performing a pre-calculation to define a complexitymetric; and distributing, based on the complexity metric, the flowsacross worker sets.

In a fourth aspect, combinable with any of the previous aspects, whereinthe complexity metric is a measure of complexities of the flow includingone or more of a number of steps in the flow or a number of specialsteps in the flow.

In a fifth aspect, combinable with any of the previous aspects, whereinspecial steps include mapping steps and scripting steps.

In a sixth aspect, combinable with any of the previous aspects, whereinthe flow represents an integration scenario.

The details of one or more implementations of the subject matter of thisspecification are set forth in the accompanying drawings and thedescription below. Other features, aspects, and advantages of thesubject matter will become apparent from the description, the drawings,and the claims.

DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram of an example environment for performing loadhandling.

FIG. 2 is a block diagram illustrating an autoscaling redistribution.

FIG. 3 is a block diagram illustrating a load-based redistribution.

FIG. 4 is a block diagram showing examples of a traditional applicationand a cloud native application (CNA).

FIG. 5 is a block diagram of an example of execution process of iFlows.

FIG. 6 is a block diagram of an example of server-to-workerinteractions.

FIG. 7 is a block diagram showing an example of an architecture thatincludes an orchestrator, a runtime, and a platform.

FIG. 8 is a block diagram showing an example of an architecture thatincludes the orchestrator, the runtime, and the platform.

FIG. 9 is a flowchart of an example of a method for reallocating flowsto worker sets.

FIG. 10 is a block diagram of an exemplary computer system used toprovide computational functionalities associated with describedalgorithms, methods, functions, processes, flows, and procedures asdescribed in the instant disclosure.

DETAILED DESCRIPTION

This disclosure generally describes computer-implemented methods,software, and systems for performing load distribution for flows. Forexample, a flow can be an iFlow that represents an integration scenario.Using techniques described in this disclosure, the number of iFlows canbe split to a manageable size for a worker, preventing the overloadingof workers. For example, new worker sets can be started, which can solvedistribution problems. Techniques are used that facilitate overcomingthe problem that physical boundaries of a worker limit executioncapabilities. Distributing iFlows to different worker sets makes itpossible to execute all of the iFlows.

An option that uses a single-iFlow container can avoid conflicts ofiFlow instances for the same resources (constant throughput problem).This is because there is simply one iFlow in a container, avoidingcompetition with other iFlows. A problem exists, however, in that thisoption can stretch the limits of the underlying technologies because ofthe number of iFlows (for example, in current implementations with 56KiFlow bundles×2 (high availability (HA))=112K).

In an option that uses a k-iFlow container, groups of iFlows can begrouped into sets of iFlows that are deployed to independent workersets. The formula means that k₁+k₂+ . . . +k_(i=n) can be used toprovide a runtime for each iFlow, where all the iFlows are deployeddirectly.

An option that uses API server/worker separation can simply split iFlowsusing microservices. The API server can expose the endpoints, and theworker can contain the pipeline execution.

FIG. 1 is a block diagram of an example environment 100 for performingload handling. The environment 100 includes a cloud platform integrationsystem 102 that operates in the cloud (for example, using a network 104)and provides integration services for work flows performed by users (forexample, using at least one client 106).

The cloud platform integration system 102 includes a load handler 108that manages relevant operations (including load balancing) and anautoscaler 110 that performs autoscaling processes. The autoscalingprocesses can include creating homogenous copies (or instances) of aparticular worker set. Thresholds are used in determining whenautoscaling should occur (versus creating a new worker setinstantiation), and can include the use of metrics to assist in thedetermination. The metrics can include metrics that are related tocentral processor unit (CPU) use (for example, “Is the CPU usage X %(for example, 80%) for a pre-determined threshold period of time”). Themetrics can also include metrics that are related to memory (forexample, “Is there N % memory usage over a pre-determined period oftime?”). CPU and memory consumption are two metrics, but the complexityof the iFlow is another metric. An iFlow's complexity can be based onbundle size (for example, in kilobytes (KB)), iFlow size (for example,the number of steps in the iFlow), and iFlow step types.

The autoscaler 110 can have limitations when, for example, too manyiFlows are deployed to a worker instance, which can require a split ofthe worker set. Thus, a combination of autoscaling and instantiating newworker sets can be used. The selection of one approach over the othercan be based on various criteria, including the number of iFlows and thenumber of worker instances.

A worker set instantiator 112 can be configured to generate a new workerset, for example, when the autoscaling alone cannot handle one or moreof the additional resources or the requirements of the new worker set.Various thresholds can be used for determining when a new worker set isto be created. For example, a first threshold can be associated with anumber of instances of a worker set (for example, five instances). Ifthe autoscaler 110 cannot distribute the load in a useful way, flows canbe switched (or reassigned) to other worker sets. A second threshold canbe established as a number of iFlows in a worker set. For example, iftoo many flows are in a worker set, the flows can be re-allocated toother existing or new worker sets. The term worker set instantiator 112can be used synonymously with or can be referred to as a contentorchestrator. The content orchestrator, for example, can trigger thestart of worker sets, for example by delegating to a tenant resourcemanager.

In some implementations, in order to achieve high-availability, workersets can be instantiated (for example, by default) with two instances ofworkers. If the autoscaler 110 increases the number of instances, itwill do this only for one worker set. The other worker sets are notimpacted, because they handle different iFlows.

An iFlow analyzer 114 can perform various tasks associated withanalyzing resources. For example, the iFlow analyzer 114 can determineresources that are needed for the execution of iFlows. The iFlowanalyzer 114 can also determine resources used during execution,including determining when a monitored load exceeds a pre-determinedthreshold. The iFlow analyzer 114 can also analyze particular resourceusage associated with iFlows and worker set instances. In someimplementations, the iFlow analyzer 114 can store (or be able to access)historical information on certain iFlows in order to obtain informationused for pre-calculation considerations and complexity metrics. Forexample, the calculation can determine a number of steps, types of iFlowsteps, and historical execution of particular iFlows. The iFlow analyzer114 can notify the worker set instantiator 112 that additional workersets are to be instantiated.

A redistribution module 116 can be configured to cause redistribution ofiFlows to occur between different worker sets. For example, based oniFlow analyzer results, the redistribution module 116 can move one ormore particular iFlows into different worker set. A load balancer 117can be used to balance loads.

A runtime environment 118 can be configured to manage one or moreinstantiated worker set instances 120, including managing the iFlows ofintegration scenario executing within the worker sets. Using a simplealgorithm, for example, the runtime environment 118 can start with onlyone worker set for use in the deployment of iFlows. Using the simplealgorithm for worker set distribution can include simply dividing thenumber of iFlows in half and distributing the iFlows equally between thetwo worker sets. A disadvantage of the simple algorithm is that there isno identification of iFlows that may cause problems (for example, usingadditional resources, based on complexity). As such, the simplealgorithm can serve primarily as a divide-and-conquer strategy forsplitting worker sets. Example splits can include [10→5+5] or [10→5+2+3](in which the second five-iFlow worker is subsequently split).Alternatively, using a pre-calculation algorithm and based on knowncharacteristics of iFlows, the runtime environment 118 can startmultiple worker sets upon deployment. The pre-calculation algorithm canuse the iFlow analyzer 114 to equally balance the complexity in theworker sets. Using the iFlow analyzer 114 can prevent an undesirablesituation in which, for example, a majority of complex iFlows areassigned to one worker set and non-complex iFlows are assigned to otherdifferent worker set.

The cloud platform integration system 102 includes memory 122 that canstore integration scenarios 124. Flow information (including iFlowdefinitions) that is defined in the integration scenarios 124 can beused by the iFlow analyzer 114 to determine static resources that areneeded based, for example, on defined flows and flow patterns. EachiFlow definition can include, for example, an associated instantiationoverhead and information about resources that are needed to execute theiFlow.

Load balancer rules 126 stored in the memory 122 can identify autoscalerthresholds and worker set generation thresholds. Historical information128 stored in the memory 122 can include information about particularflows including, for example, resource consumption and other statisticalinformation for particular flows.

The cloud platform integration system 102 includes an interface 130 thatprovides interface services to clients 106. The cloud platformintegration system 102 also includes a processor 132 that can executeinstructions of modules and components of the cloud platform integrationsystem 102.

The cloud platform integration system 102 can be used for real-timedistribution of iFlows at runtime so that resources can be used in amore efficient way. Considerations regarding efficiency can be explainedusing examples illustrating extreme conditions. In an un-optimizedextreme case, all iFlows can be run on one runtime (for example, ann-iFlow case), which can create the problem, for example, of overloadingan Open Service Gateway Initiative (OSGI) runtime with the number ofbundles needed for the iFlows. In this example, a same-time execution ofall iFlows can be potentially done on the same runtime, creating aresource sharing problem. Therefore, in considering efficiency,techniques for k-iFlow solutions can be used, where 1>k>n.

Each iFlow can be categorized based on the iFlow's characteristics. Forexample, in a static case, before an iFlow is deployed, the iFlow bundlefor the iFlow can be analyzed regarding the iFlow's complexity. Criteriafor categorizing the iFlow's complexity can include, for example, bundlesize (for example, in kilobytes (KB)), iFlow size (for example, thenumber of steps in the iFlow), and iFlow step types. When consideringiFlow step types, an assumption is that splitter, mapping, and scriptsteps create significant memory and CPU usage.

In a dynamic case, for example, analyzing an iFlow's load after theiFlow is deployed, it is possible to measure the iFlow's usage ofresources in real time. Measurements can be made, for example, of CPUuse, memory use, a number of requests (per iFlow), the size of therequests, and out-of-memory conditions, as exceptions can indicate thata particular iFlow on the runtime is creating problems.

Various algorithm approaches can be used in the distribution of iFlows.A first algorithm approach, for example, can start with a staticanalysis and fill a worker based on the static criteria optimized basedon the dynamic measurements. This approach includes the challenge ofredistribution of iFlows from one worker to another one. The pros ofthis approach include an optimized use of resources and subsequentredistribution of iFlows. A con of this approach includes the need for acomplex redistribution algorithm.

A second algorithm approach, for example, can start with a staticanalysis and fill a worker based on the static criteria. If the workerset is full, a next worker set can be used, but there is no dynamicredistribution. The challenge in this approach is to define the staticcriteria in a good way to handle the resource consumption. The pros ofthis approach include a solution for the problem of number of OSGIbundles (because only a limited number of iFlows are running at the sametime on a runtime) and the complexity to redistribute iFlows is low.Cons of this approach include the lack of a solution to the resourceconsumption problem (because all iFlows are deployed to handle requests)and there is no redistribution of iFlows).

Different ways for calculating resource usage can depend on the numberof iFlows. The single-iFlow approach and the n-iFlow approach canprovide two extremes, and the k-iFlow approach can provide a compromise.Advantages of the k-iFlow approach can become obvious, because with kiFlows it is possible to run a k-iFlow as a single-iFlow (running eachiFlow in its own worker set) or an n-iFlow (only running one workerset). In the examples that follow, the number of iFlows is 400, runtimememory consumption is 1000 MB, iFlow memory consumption is 1 megabyte(MB), and the number of worker sets is 10.

In the single-iFlow approach, memory consumption can be, for example,400,400 MB. The single-iFlow approach has the problem that the fixedmemory consumption of runtime can lead to very high memory consumptionoverall. The number of workers is the number of iFlows.

In the n-iFlow approach, memory consumption can be, for example, 1400MB. The n-iFlow approach can have the following disadvantages. First, itis not possible to deploy more than 200 iFlows to a worker. Second,because iFlows run together at one runtime, an exception in one iFlowcan crash all of the other iFlows. However, memory consumption(including reuse of runtime) can be efficient.

In the k-iFlow approach, memory consumption can be, for example, 10,400MB, and memory consumption of one worker can be 1040 MB. The k-iFlowapproach can serve as a compromise between the n-iFlow approach (whichcan create a runtime problem with the number of iFlows) and memoryconsumption. The number of workers can be defined by the number ofworker sets (which can be dynamically configurable).

FIG. 2 is a block diagram illustrating an autoscaling redistribution200. The autoscaling redistribution 200 can solve a problem in whichworkers are under a heavy load (for example, iFlows 202 a-202 d in aworker set 204). A worker set under a 90% load 206 can be split, usingautoscaling 208, into worker sets 210 and 212, each having a 45% load214. In this example, half of the iFlows 202 a-202 d end up in each ofthe worker sets 210 and 212. However, autoscaling 208 cannot handlecomplex redistribution.

FIG. 3 is a block diagram illustrating a load-based redistribution 310.Before redistribution, iFlows 202 a-202 f are distributed in worker sets302, 304, and 306, which include three, two and one iFlow, respectively.Also, iFlow_2 202 b and iFlow_3 202 c are each determined to have a 45%load 308. In this example, the worker set 302 has a 90% load 310 a (aheavy load), and the worker sets 304 and 306 each have a 0% load 310 b.Under redistribution 310, the iFlow_3 202 c can be moved from the workerset 302 to the worker set 306. As a result, the worker sets 302 and 306can have a 45% load 310 c, while the worker set 304 still has a 0% load310 b.

Redistribution 310 may not always be deterministic in determining whichiFlow should be moved to another worker set and can lead to sub-optimalredistribution decisions with local optima. For example, afterredistribution 310, iFlow_4 202 d and iFlow_5 202 e still do not haveoptimal resource utilization.

Various options can be used to improve the effectiveness ofredistribution. In a first redistribution option, an applicationprogramming interface (API) server can be used to analyze theutilization of iFlows and a current allocation of iFlows to worker sets.A consequence is that unused iFlows can be un-deployed. Efficientredistribution can require an algorithm or framework to identify theiFlow that is under heavy load. The redistribution algorithm needs to beable to start the workers with changed sets of iFlows, and this canfurther increase the complexity of the API server.

In a second redistribution option without using an API server,redistribution algorithms still need to analyze the load of the iFlowsand identify used iFlows. Challenges created without the use of the APIserver include relying on platform capabilities for dispatching. Forexample, platform capabilities have to be used to achieve a stableuniform resource locator (URL) for consumers of the endpoints. Smartredistribution is required to identify iFlow that are under heavy loadand provide iFlows to different sets of workers.

FIG. 4 is a block diagram showing examples of a traditional application400 a and a cloud native application (CNA) 400 b. A traditionalapplication 402 includes a user interface (UI) 404 that serves as afront-end to users. Flows are performed in a runtime environment 406using a persistent state 408 of the work. In a traditional application(app) 410, size is actively managed 412, which can causeoverprovisioning of resources and monolithic deployments 414. A state iskept 416 for the whole application. In-process communication 418 occurs,and implementation 420 occurs for projects and operations (ops).

The worker node can need to be scaled manually, leading tooverprovisioning of resources to fulfill each request. Types ofsituations that stretch resources include, for example, peak (and verybig) messages and constant throughput. In the case of peak (very big)messages, cloud platform integrations (CPIs) can get a 100 megabyte (MB)extensible markup language (XML) message. XML serialization can resultin an explosion of used memory (assuming a 500 MB used heap size). If aniFlow contains a mapping, then an average factor of 6× can apply tomemory consumption to transform the message from one format to the other(for example, 500 MB×6=3000 MB). In the case of constant throughput, theCPI can receive a lot of messages, and mapping and scripting can createa significant memory requests. When an out-of-memory-exception occurs,memory crashes can occur.

A CNA 422 is elastically scalable 424 and has a microservices-baseddeployment 426. States of flows are isolated (428) in a few components,and loosely coupled by contract, where remote communication (430) occursover a network. Deployment can be automated (432) for developmentoperations (DevOps) and NoOps, including automation of deployment,monitoring, and management of iFlows.

A CNA 434 includes a UI 436 that serves as a front-end to users. Flowsare facilitates using microservices 438-442 using persistent states 444and 446. Reliability equations 448 and 450 applicable to the traditionalapp 410 and the CNA 422 can be based on mathematical relationshipsbetween a mean time to failure (MTTF) and a mean time to replacement(MTTR).

Various options can be used for managing iFlow containers. In astreaming option, a pipeline of iFlows can work with streaming to reducethe memory consumption.

In an option that uses splitting into microservices, a CNA architecturecan recommend to split a runtime into smaller units to handle a loadwith smaller units. Because the mapping engine is focused on performingmappings, the mapping engine does not require so much memory. A peakscenario can potentially fit to this option, because a complete iFlowdoes not have to be held in memory. However, without streaming, thememory consumption of the mapping engine can be in the same dimension.

FIG. 5 is a block diagram of an example of execution process 500 ofiFlows. The execution process 500 covers a single-iFlow worker that issimilar to the single-iFlow container implementations and in which theworker is only executing one iFlow. A user 502 performing tasks caninterface with APIs 504 that provide endpoints 506, a runtime 508, andplatform specifics 510. The APIs 504 can facilitate execution andmanagement of a task 512 that is under the control of a worker instance514. The worker instance 514 provides an integration flow 516, a runtime518, and platform specifics 520. The APIs 504 and the worker instances514 can handle messages 522 that are created during an iFlow.

TABLE 1 Per-container Size Parameters and Conditions One iFlow K iFlowsN per container per container iFlows Without With With K-iFlows/ perAPI- API- With Without redistribution container with container ServerServer redistribution redistribution and API-Server assignment Large no.of No Yes Yes Yes Yes Yes Possible iFlows Cost Low High Medium MediumHigh/ Medium Low to High Medium Isolation of No Yes Yes Partial NoPartial Possible iFlows Complexity Low Low Medium High Medium High LowDependency Medium Medium Medium High Medium High Medium to Platform

Some architecture decisions can be based on an evaluation of the APIserver/worker split with a single-iFlow worker set. The single-iFlowworker set can provide a fair compromise between complexity and resourceoptimization, assuming the following: 1) a warmed-up container pool isneeded, as dynamic startup of containers would be too time-consuming; 2)a risk still exists, depending on the usage, that the cost could go up,as the resource sharing across containers is quite low (where the usageof iFlows per tenant is <50%); 3) the API server can use the loadbalancer for dispatching, allowing the load balancing, failover, andautoscaling to be handled by the platform; 4) for timer-triggered iFlows(typically a rare type of iFlow), the container can always be started;5) it may not be practical to combine the k-iFlow approach with dynamicdistribution due to differing sets of complex challenges; 6) the APIserver has the potential to be a single point of failure (depending onthe deployment topology), which can adversely affect scalability andreliability; and 7) it may be technically possible that multipleintegration flows expose the same end-point.

In some implementations, splitting of iFlows can occur. In conventionalsystems, an iFlow can be transformed from a beans.xml packaged into anOSGI bundle. The split implies that the endpoint has to be pushed to theAPI server and the remaining iFlow has to go to the worker.

The iFlows can be categorized into three groups: 1) groups usinghypertext transfer protocol (HTTP) endpoints, used for on-demandexecution of iFlows; 2) groups using fast scheduling, in which adaptersare polling regularly, and if new data exists, processing has to bestarted, for example, using file transfer protocol (FTP) or Java messageservice (JMS)); and 3) groups using irregular scheduling, includingrun-once or specific scheduling mechanisms.

Technology for splitting iFlows can include the following options. In afirst option, API servers can reuse the same beans.xml. The beans.xmlcan be deployed to the API server, where only the consumer is executedbecause the route definition is changed dynamically.

A second option can use an API server with special beans.xml used forendpoint. In this option, iFlow generation can create two bundles: 1)beans.xml with a consumer endpoint and a delegate-to-worker endpoint;and 2) beans.xml with the remaining functionality of the iFlow.

A third option can use a Tomcat-API server, and iFlow generation cancreate an artifact in which the Tomcat-API server provides separateservlets for each iFlow and metadata to handle redirection of iFlows toworkers (generic redirects at runtime).

FIG. 6 is a block diagram of an example of server-to-workerinteractions. An API server 602 can reuse a runtime to expose endpointsthrough at least one route 604 (and through a route 606 in a worker node608). There can be a specific endpoint bundle that is executed on theAPI server. A pro is that scheduling adapters can be executed on the APIserver and no worker has to be started for polling operations. Consinclude: a footprint of runtime to expose endpoints is higher than aTomcat runtime, and transaction handling can become more complex forsome adapters, for example, JMS and applicability statement 2 (AS2).Based on this information, implementations can use a light-weight APIserver to avoid the split of the beans.xml, and the route can runcompletely on the worker. Selected implementations can also be based onhow the API server identifies the worker that will take over the work,including deploying the iFlow to the worker and evicting the iFlow incase there are no requests coming any more.

The selection of an architecture can be based, for example, on discoverybeing merged with the orchestrator functionality, because theorchestrator is responsible for distribution of bundles and lifecycle ofnodes, e.g., the deployment of iFlow bundles to workers, therefore thestate of workers and iFlows bundles is available/stored at the nodeorchestrator. As such, options for orchestrator can include thefollowing.

The orchestrator can be considered one actor. However, evaluation candetermine that the orchestrator has at least two purposes: nodemanagement and content management. Regarding node management, theorchestrator is responsible for all lifecycle operations of nodes,including API servers. The orchestrator is responsible for launching andstopping applications/pods. The orchestrator gathers information aboutthe health of each application instance single actor which queries thenumber of applications running the health of each application. Theorchestrator mounts file system volumes for each application/pod. Theorchestrator defines and enforces security and network policies for eachapplication/pod. The orchestrator provides binding of services requiredby each application and cleanup of resources in case of stop ofapplication/pod.

In some implementations, the orchestrator can be split into two maindomains based on the dimension of tenancy (for example, multi-tenantnode or tenant-specific). For example, four options can exist for theorchestrator: 1) multi-tenant node orchestrator, and multi-tenantcontent orchestrator; 2) multi-tenant node orchestrator, andtenant-specific content orchestrator; 3) tenant-specific nodeorchestrator, and multi-tenant content orchestrator; and 4)tenant-specific node orchestrator, and tenant-specific contentorchestrator.

FIG. 7 is a block diagram showing an example of an architecture 802 athat includes an orchestrator 802, a runtime 804, and a platform 806. Atenant resource manager 812 of the orchestrator 802 is responsible forthe resource utilization and management of workers, shares, anddatabases. A content orchestrator 810 of the orchestrator 802 can handlethe content for a particular tenant. The result, in the architecture 800a, is that tenant resource manager 812 is multi-tenant and the contentorchestrator 810 is tenant-specific.

FIG. 8 is a block diagram showing an example of an architecture 800 bthat includes the orchestrator 802, the runtime 804, and the platform806. In the architecture 800 b, the content orchestrator 810 is assignedto a microservice of a runtime 804 to handle iFlow bundles for iFlows820 of worker instances 818.

The goal of a separation between an API server 814 and the workerinstances 818 is that workers are only started if there are requests fora particular iFlow. This implies that there has to be a management ofworker node instances that includes starting of new workers (if requestsfor iFlow instances not already used) and stopping workers that are notused within a predetermined time frame. It can be assumed that there isa pool of workers that are available for dispatching. In case a requestcomes to the API server 814, the worker gets the corresponding iFlow, amessage is redirected, and execution can be handled.

The API server 814 handles the lifecycle management operations. Anadvantage is that the API server 814 handles the workers and iFlowscompletely. A disadvantage is that there is overlap with functionalityof content orchestrator 810. The content orchestrator 810 can be calledevery time a lifecycle operation is required. An advantage is that theorchestrator 802 has the clear responsibility of handling workers andiFlows. A disadvantage is that the API server 814 has to contact theorchestrator 802 regarding crossing boundaries of microservices andpotentially violating the deployment of the runtime 804. The API server814 can call the content orchestrator 810 to start new workers and tostop unused workers.

FIG. 8C is a block diagram showing an example of an architecture 800 cthat includes the orchestrator 802, the runtime 804, and the platform806. In the architecture 800 c, iFlows can be deployed directly toworker nodes, and all iFlows can be deployed to all worker nodes of atenant. In implementations using the single-iFlow container, when aniFlow is deployed to a worker with the dynamic decomposition, it is alsorequired that the iFlow is deployed at a later point in time to theworker.

The API server 814 can trigger the deployment of the iFlow to a worker.In this example, the orchestrator 802 can become a potential bottleneckfor deployments, worker node retrieves bundle. Further, worker nodes canget bundles from a repository and deploy bundles without contacting anyother node. The worker alone can handle deployment of iFlow, and theworker needs to take over content lifecycle tasks.

Message execution and communication can be delegated to the worker. Insome implementations, messages can be synchronous using HTTP. Messagescan exist in an object store. Based on the message size, a message canbe transferred directly or using the object store, which can providebetter resource utilization and thread management. In someimplementations, messages can be sent asynchronously using a queue(implying the use of the object store). Messaging can rely completely onthe use of the object store. However, based on the message size, themessage can be transferred directly. The messaging techniques that areused can depend on considerations of reliability, latency, and the useof a message broker or an object store.

FIG. 9 is a flowchart of an example of a method 900 for reallocatingflows to worker sets. Method 900 can be performed by the cloud platformintegration system 102, for example. For clarity of presentation, thedescription that follows generally describes method 900 in the contextof FIGS. 1 through 8C.

At 902, a set of flows is initially deployed to a single worker set. Asan example, the iFlows 202 a-202 d can be deployed to the worker set204.

At 904, a load balancing issue is identified that is associated withinitial runtime interactions by workers with the single worker set. Forexample, the load handler 108 can determine that the worker set 204 hasa heavy load (for example, 90%).

At 906, in response to identifying the load balancing issue, the loadbalancing issue is analyzed to determine whether to autoscale orgenerate a new worker set. The load handler 108 can determine, forexample, whether the worker set 204 should be autoscaled (for example,into worker sets 210 and 212) or new worker sets should be generated bythe worker set instantiator 112.

At 908, load balancing is performed to initiate at least one new workerset. For example, the load handler 108 can create new worker sets 210and 212 that are needed to balance the load.

At 910, at least one flow to be moved to the at least one new worker setis identified. For example, the iFlows 202 a-202 d can be split by loadhandler 108 among the new worker sets 210 and 212.

In some implementations, identifying at least one flow to be moved tothe at least one new worker set includes randomly moving the at leastone flow. For example, random ones of the iFlows 202 a-202 d can beselected by load handler 108 to be moved to the new worker sets 210 and212.

In some implementations, identifying at least one flow to be moved tothe at least one new worker set can includes performing apre-calculation to define a complexity metric (for example, determininga complexity of iFlows 202 a-202 e that are candidates to bere-allocated). For example, complexity can be based on one or more of anumber of steps in the flow or a number of special steps in the flow.Special steps can include, for example, mapping steps and scriptingsteps. The flows can be distributing across worker sets based on thecomplexity metric. For example, the complexity of the iFlows 202 a-202 ecan determine which of the worker sets 302-306 are to contain specificones of the iFlows 202 a-202 e.

At 912, movement of the identified at least one flow from a currentworker set to a new worker set is performed. As an example, referring toFIG. 3, the load handler 108 can move iFlow_3 202 c to the worker set306.

FIG. 10 is a block diagram of an exemplary computer system 1000 used toprovide computational functionalities associated with describedalgorithms, methods, functions, processes, flows, and procedures asdescribed in the instant disclosure.

The illustrated computer 1002 is intended to encompass any computingdevice such as a server, desktop computer, laptop/notebook computer,wireless data port, smart phone, personal data assistant (PDA), tabletcomputing device, one or more processors within these devices, or anyother suitable processing device, including both physical or virtualinstances (or both) of the computing device. Additionally, the computer1002 may comprise a computer that includes an input device, such as akeypad, keyboard, touch screen, or other device that can accept userinformation, and an output device that conveys information associatedwith the operation of the computer 1002, including digital data, visual,or audio information (or a combination of information), or a graphicaluser interface (GUI).

The computer 1002 can serve in a role as a client, network component, aserver, a database or other persistency, or any other component (or acombination of roles) of a computer system for performing the subjectmatter described in the instant disclosure. The illustrated computer1002 is communicably coupled with a network 1030. In someimplementations, one or more components of the computer 1002 may beconfigured to operate within environments, includingcloud-computing-based, local, global, or other environment (or acombination of environments).

At a high level, the computer 1002 is an electronic computing deviceoperable to receive, transmit, process, store, or manage data andinformation associated with the described subject matter. According tosome implementations, the computer 1002 may also include or becommunicably coupled with an application server, e-mail server, webserver, caching server, streaming data server, business intelligence(BI) server, or other server (or a combination of servers).

The computer 1002 can receive requests over network 1030 from a clientapplication (for example, executing on another computer 1002) andresponding to the received requests by processing the said requests inan appropriate software application. In addition, requests may also besent to the computer 1002 from internal users (for example, from acommand console or by other appropriate access method), external orthird-parties, other automated applications, as well as any otherappropriate entities, individuals, systems, or computers.

Each of the components of the computer 1002 can communicate using asystem bus 1003. In some implementations, any or all of the componentsof the computer 1002, both hardware or software (or a combination ofhardware and software), may interface with each other or the interface1004 (or a combination of both) over the system bus 1003 using an API1012 or a service layer 1013 (or a combination of the API 1012 andservice layer 1013). The API 1012 may include specifications forroutines, data structures, and object classes. The API 1012 may beeither computer-language independent or dependent and refer to acomplete interface, a single function, or even a set of APIs. Theservice layer 1013 provides software services to the computer 1002 orother components (whether or not illustrated) that are communicablycoupled to the computer 1002. The functionality of the computer 1002 maybe accessible for all service consumers using this service layer.Software services, such as those provided by the service layer 1013,provide reusable, defined business functionalities through a definedinterface. For example, the interface may be software written in JAVA,C++, or other suitable language providing data in extensible markuplanguage (XML) format or other suitable format. While illustrated as anintegrated component of the computer 1002, alternative implementationsmay illustrate the API 1012 or the service layer 1013 as stand-alonecomponents in relation to other components of the computer 1002 or othercomponents (whether or not illustrated) that are communicably coupled tothe computer 1002. Moreover, any or all parts of the API 1012 or theservice layer 1013 may be implemented as child or sub-modules of anothersoftware module, enterprise application, or hardware module withoutdeparting from the scope of the instant disclosure.

The computer 1002 includes an interface 1004. Although illustrated as asingle interface 1004 in FIG. 10, two or more interfaces 1004 may beused according to particular needs, desires, or particularimplementations of the computer 1002. The interface 1004 is used by thecomputer 1002 for communicating with other systems in a distributedenvironment that are connected to the network 1030 (whether illustratedor not). Generally, the interface 1004 comprises logic encoded insoftware or hardware (or a combination of software and hardware) andoperable to communicate with the network 1030. More specifically, theinterface 1004 may comprise software supporting one or morecommunication protocols associated with communications such that thenetwork 1030 or interface's hardware is operable to communicate physicalsignals within and outside of the illustrated computer 1002.

The computer 1002 includes a processor 1005. Although illustrated as asingle processor 1005 in FIG. 10, two or more processors may be usedaccording to particular needs, desires, or particular implementations ofthe computer 1002. Generally, the processor 1005 executes instructionsand manipulates data to perform the operations of the computer 1002 andany algorithms, methods, functions, processes, flows, and procedures asdescribed in the instant disclosure.

The computer 1002 also includes a memory 1006 that holds data for thecomputer 1002 or other components (or a combination of both) that can beconnected to the network 1030 (whether illustrated or not). For example,memory 1006 can be a database storing data consistent with thisdisclosure. Although illustrated as a single memory 1006 in FIG. 10, twoor more memories may be used according to particular needs, desires, orparticular implementations of the computer 1002 and the describedfunctionality. While memory 1006 is illustrated as an integral componentof the computer 1002, in alternative implementations, memory 1006 can beexternal to the computer 1002.

The application 1007 is an algorithmic software engine providingfunctionality according to particular needs, desires, or particularimplementations of the computer 1002, particularly with respect tofunctionality described in this disclosure. For example, application1007 can serve as one or more components, modules, applications, etc.Further, although illustrated as a single application 1007, theapplication 1007 may be implemented as multiple applications 1007 on thecomputer 1002. In addition, although illustrated as integral to thecomputer 1002, in alternative implementations, the application 1007 canbe external to the computer 1002.

There may be any number of computers 1002 associated with, or externalto, a computer system containing computer 1002, each computer 1002communicating over network 1030. Further, the term “client,” “user,” andother appropriate terminology may be used interchangeably as appropriatewithout departing from the scope of this disclosure. Moreover, thisdisclosure contemplates that many users may use one computer 1002, orthat one user may use multiple computers 1002.

In some implementations, components of the environments and systemsdescribed above may be any computer or processing device such as, forexample, a blade server, general-purpose personal computer (PC),Macintosh, workstation, UNIX-based workstation, or any other suitabledevice. In other words, the present disclosure contemplates computersother than general purpose computers, as well as computers withoutconventional operating systems. Further, components may be adapted toexecute any operating system, including Linux, UNIX, Windows, Mac OS®,Java™, Android™, iOS or any other suitable operating system. Accordingto some implementations, components may also include, or be communicablycoupled with, an e-mail server, a web server, a caching server, astreaming data server, and/or other suitable server(s).

Processors used in the environments and systems described above may be acentral processing unit (CPU), an application specific integratedcircuit (ASIC), a field-programmable gate array (FPGA), or anothersuitable component. Generally, each processor can execute instructionsand manipulates data to perform the operations of various components.Specifically, each processor can execute the functionality required tosend requests and/or data to components of the environment and toreceive data from the components of the environment, such as incommunication between the external, intermediary and target devices.

Components, environments and systems described above may include amemory or multiple memories. Memory may include any type of memory ordatabase module and may take the form of volatile and/or non-volatilememory including, without limitation, magnetic media, optical media,random access memory (RAM), read-only memory (ROM), removable media, orany other suitable local or remote memory component. The memory maystore various objects or data, including caches, classes, frameworks,applications, backup data, business objects, jobs, web pages, web pagetemplates, database tables, repositories storing business and/or dynamicinformation, and any other appropriate information including anyparameters, variables, algorithms, instructions, rules, constraints, forreferences thereto associated with the purposes of the target,intermediary and external devices. Other components within the memoryare possible.

Regardless of the particular implementation, “software” may includecomputer-readable instructions, firmware, wired and/or programmedhardware, or any combination thereof on a tangible medium (transitory ornon-transitory, as appropriate) operable when executed to perform atleast the processes and operations described herein. Indeed, eachsoftware component may be fully or partially written or described in anyappropriate computer language including C, C++, Java™, Visual Basic,assembler, Perl®, any suitable version of 4GL, as well as others.Software may instead include a number of sub-modules, third-partyservices, components, libraries, and such, as appropriate. Conversely,the features and functionality of various components can be combinedinto single components as appropriate.

Devices can encompass any computing device such as a smart phone, tabletcomputing device, PDA, desktop computer, laptop/notebook computer,wireless data port, one or more processors within these devices, or anyother suitable processing device. For example, a device may comprise acomputer that includes an input device, such as a keypad, touch screen,or other device that can accept user information, and an output devicethat conveys information associated with components of the environmentsand systems described above, including digital data, visual information,or a GUI. The GUI interfaces with at least a portion of the environmentsand systems described above for any suitable purpose, includinggenerating a visual representation of a web browser.

The preceding figures and accompanying description illustrate exampleprocesses and computer implementable techniques. The environments andsystems described above (or their software or other components) maycontemplate using, implementing, or executing any suitable technique forperforming these and other tasks. It will be understood that theseprocesses are for illustration purposes only and that the described orsimilar techniques may be performed at any appropriate time, includingconcurrently, individually, in parallel, and/or in combination. Inaddition, many of the operations in these processes may take placesimultaneously, concurrently, in parallel, and/or in different ordersthan as shown. Moreover, processes may have additional operations, feweroperations, and/or different operations, so long as the methods remainappropriate.

In other words, although this disclosure has been described in terms ofcertain implementations and generally associated methods, alterationsand permutations of these implementations, and methods will be apparentto those skilled in the art. Accordingly, the above description ofexample implementations does not define or constrain this disclosure.Other changes, substitutions, and alterations are also possible withoutdeparting from the spirit and scope of this disclosure.

What is claimed is:
 1. A computer-implemented method comprising:deploying a set of flows initially to a single worker set; identifying aload balancing issue associated with initial runtime interactions byworkers with the single worker set; in response to identifying the loadbalancing issue, analyzing the load balancing issue to determine whetherto autoscale or generate a new worker set; performing load balancing toinitiate at least one new worker set; identifying at least one flow tobe moved to the at least one new worker set, wherein identifying the atleast one flow to be moved to the at least one new worker set comprisesrandomly moving at least one flow to the at least one new worker set;and performing movement of the identified at least one flow from acurrent worker set to a new worker set.
 2. The computer-implementedmethod of claim 1, wherein the load balancing issues includeout-of-memory exceptions.
 3. The computer-implemented method of claim 1,wherein the flow represents an integration scenario.
 4. A systemcomprising: memory storing tables of information for integrationscenarios, load balancer rules, and historical information; and a serverperforming operations comprising: deploying a set of flows initially toa single worker set; identifying a load balancing issue associated withinitial runtime interactions by workers with the single worker set; inresponse to identifying the load balancing issue, analyzing the loadbalancing issue to determine whether to autoscale or generate a newworker set; performing load balancing to initiate at least one newworker set; identifying at least one flow to be moved to the at leastone new worker set, wherein identifying the at least one flow to bemoved to the at least one new worker set comprises randomly moving atleast one flow to the at least one new worker set; and performingmovement of the identified at least one flow from a current worker setto a new worker set.
 5. The system of claim 4, wherein load balancingissues include out-of-memory exceptions.
 6. The system of claim 4,wherein the flow represents an integration scenario.
 7. A non-transitorycomputer-readable media encoded with a computer program, the programcomprising instructions that when executed by one or more computerscause the one or more computers to perform operations comprising:deploying a set of flows initially to a single worker set; identifying aload balancing issue associated with initial runtime interactions byworkers with the single worker set; in response to identifying the loadbalancing issue, analyzing the load balancing issue to determine whetherto autoscale or generate a new worker set; performing load balancing toinitiate at least one new worker set; identifying at least one flow tobe moved to the at least one new worker set, wherein identifying the atleast one flow to be moved to the at least one new worker set comprisesrandomly moving at least one flow to the at least one new worker set;and performing movement of the identified at least one flow from acurrent worker set to a new worker set.
 8. The non-transitorycomputer-readable media of claim 7, wherein the load balancing issuesinclude out-of-memory exceptions.
 9. The non-transitorycomputer-readable media of claim 7, wherein the flow represents anintegration scenario.